Neural network is a large-scale multi-parameter optimization tool. With a large amount of training data, neural network can learn hidden features that are difficult to summarize in the data, thus completing many complex tasks, such as face detection, image semantic segmentation, object detection, motion tracking, and natural language translation. Neural network has been widely used in the artificial intelligence community.
In image processing applications such as modern deep visual recognition, the Convolutional Neural Network (CNN) model is at the core. However, since the convolutional network usually requires a huge amount of computation, the Convolutional Neural Network model must be simplified in order to be applied to application scenarios with low power consumption such as mobile devices. In the existing optimization schemes of Convolutional Neural Network used for image processing, some schemes only use information of the convolutional kernel itself without taking advantage of characteristics of input data distribution at each layer, some schemes use Stochastic Gradient Descent (SGD) that is instable in certain situations and has a risk of falling into a local extremum as the basic algorithm for optimization, some other schemes only use certain simple statistical information of channels (such as mean, variance, etc.) without considering interactive effects among channels, besides, some schemes require retraining on the optimized model, the optimization process is time-consuming and makes parameter tuning inconvenient. In other words, the existing schemes generally have the disadvantages of low optimization efficiency, serious loss of accuracy, and very limited speed-up ratio.